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Abstract 

In this work we present a simplifyed proof of Kantorovich's Theorem 
on Newton's Method. This analysis uses a technique which has already 
been used for obtaining new extensions of this theorem. 
AMSC: 49M15, 90C30. 

1 Introduction 

Kantorovich's Theorem assumes semi-local conditions to ensure existence and 
uniqueness of a solution of a nonlinear equation F(x) — 0, where F is a dif- 
ferentiable application between Banach spaces [SJ [HI El [H] • This theorem uses 
constructively Newton method and also guarantee convergence to a solution of 
this iterative procedure. Apart from the elegance of this theorem, it has many 
theoretical and practical applications, in [ 10] we can find a reviews of recent 
applications and in jll) an application in interior point methods. This theorem 
has also many extensions, some of then encompassing previously unrelated re- 
sults see HIES- Some of these generalizations and extensions are quite recent, 
because in the last few year the Kantorovich's Theorem has been the subject of 
intense research, see [H |2) |3) [T0 l \U \ H3] . 

The aim of this paper is to present a new technique for the analysis of the 
Kantorovich's Theorem. This technique, was introduced in [2i and since then it 
has been used for obtaining new extensions of Kantorovich's Theorem see [TJ [3] . 
Here, it will be used to present a simplified proof of its "classical" formulation. 

The main idea is to define "good" regions for Newton method, by comparing 
the nonlinear function F with its scalar majorant function. Once these good 
regions are obtained, an invariant set for Newton method is also obtained and 
there, Newton iteration can be repeated indefinitely. 
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The following notation is used throughout our presentation. Let X be a 
Banach space. The open and closed ball at x € X are denoted, respectively by 

B(x, r) — {y £ X; \\x — y\\ < r} and B[x, r] = {y £ X; \\x — y\\ ^ r}. 

For the Frechet derivative of a mapping F we use the notation F' and for the 
Dual space of X we use X*. 

First, let us recall Kantorovich's theorem on Newton's method in its classical 
formulation, see H © [SJ GO [TT] . 

Theorem 1. Let X , Y be Banach spaces, CCI and F : C — > Y a continuous 
function, continuously differ •entiable on int(C). Take xq € int(C), L. b > and 
suppose that 

1) F (xq) is non-singular, 

2) \\F , (x )- 1 [F'(y)-F'(x)]\\<L\\x-y\\ foranyx,yeC, 

3) \\F'(xo)- x F(x )\\ <b, 

4) 2bL < 1. 
Define 

_ 1 - VI - 2bL 1 + VI - 2bL 

// 

B[x ,t r ] c C, 

then the sequences {xk} generated by Newton's Method for solving F(x) = 
with starting point Xq, 

x k+1 =Xk-F'(x k )- 1 F(xk), k = 0,l,---, (2) 

is well defined, is contained in B(xQ,t*), converges to a point x* € B[xo,t*] 
which is the unique zero of F in _B[xo,i*] and 

\\x* - Xk+i\\ < ^\\ x * - x kh k — 0, 1, (3) 

Moreover, if assumption 4 holds as an strict inequality, i.e. 2bL < 1, then 

1 — 9 2k L L 
< 1 + ^ 2V1-26L 1 '"" 2 " 2Vl-2^ ' |a: *"" fcl12 ' * = °' 1; ' ' ' 

(4) 

where 9 :— t*/t** < 1, and x* is the unique zero of F in B[xq, p] for any p such 
that 

t* < p < £**, B[xq, p] C C. 

Note that under assumption 1-4, convergence of {x^ is Q-linear, according to 
The additional assumption 2bL < 1 guarantee Q-quadratic convergence, 
according to (QJ. This additional assumption also guarantee that is the 
unique zero of F in B(xo,t**), whenever £?(xo,f**) C C. 

From now on, we assume that the hypotheses of Theorem Q] hold, with 
the exception of 2bL < 1 which will be considered to hold only when explicitly 
stated. 
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2 Kantorovich's Theorem for a scalar quadratic 
function 

In this section we analyze Newton method applied to solve the scalar equation 
f(t) = 0, for / : R -> K 

f(t) = ^t 2 -t + b. (5) 

The analysis to be performed can also be viewed as Kantorovich's theorem for 
function /. This function and the sequence generated by Newton method for 
solving /(f) = with starting point to, 

to := 0, (6) 

both will play an important rule in the analysis of Theorem [TJ 

Note that the assumptions of Theorem [JJ are satisfied in the very particular 
case F = /, X = Y = C = R, x = t . The roots of / are t* and t*», as defined 
in ([TJ. As b, L > 0, 

*\ t%%^ 

with strict inequality between i* and f** if and only if 2bL < 1. Hence 

• f» is the unique root of / in £>[to,t»], 

• if 2bL < 1, then t„ is the unique root of / in i?(to,t**). 

So, the existence and uniqueness part of Theorem Q] for zeros of / holds. 

Proposition 2. The scalar function f has a smallest nonnegative root t» € 
(0, 1/L]. Moreover, for any t 6 [0,t«) 

/(t)>0, f'(t)<L(t-U)<0. 

Proof. For the first statement, it remains to prove that i* < 1/L, which is a 
trivial consequence of the assumptions on b and L. 

As /(0) > 0, / shall be strictly positive in [0,t»). For the last inequalities, 
use the inequality t* < 1/L and ([5]) to obtain 

/'(t) = it - 1 = L(t - 1/L) < L(t - U). 

Now, the last inequality follows directly from the assumption t < t». □ 

According to Proposition^ f'(t) ^ for all t € [0,t„). Therefore, Newton 
iteration is well defined in [0,t*). Let us call it nf, 

n f :[0,U) ^ R 

t ^ t-/(t)//'(t). 10 

Note that, up to now, only one single iteration of newton method is well defined 
in [0,t*). In principle, Newton iteration could map some t € [0,t„) in to 1/L. 
In such a case, the second iterate for t would be not defined. 

Now, wc shall prove that Newton iteration can be repeated indefinitely at 
any starting point in [0,t*). 
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Proposition 3. For any t € [0,f„) 

t*-n f (t) = -j—{U-tf, t<n f {t)<U. 
In particular, rif maps [0,t*) in [0,t*). 

Proof. Take t € [G,i*). As / is a second-degree polynomial and /(£*) = 0, 

o = + /'(«)(** -*) + §(«*- *) 2 - 

Dividing by /'(£) we obtain, after direct rearranging 

U-t + f{t)/f\t) = -^- ) {U-tf. 

Note that, by (J7J, the left hand side of the above equation is i* — n/(t), which 
proves the first equality. 

Using Proposition [2] we have / (t) > and /'(i) < 0. Combining these 
inequalities with definition (0 and the first equality in the proposition, respec- 
tively, we obtain t < rif(t) < t*. The last statement of the Proposition follows 
directly from these inequalities. □ 

Proposition [3] shows, in particular, that for any t € [0, £*), the sequence 
{n k f (t)}, 

n° f (t) = t, n) +1 {t) =n f (n k f (t)) , fe = 0,l,---. 

is well defined, strictly increasing, remains in [0, £*) and so, is convergent. There- 
fore, Newton method for solving f(t) =0 with starting point to = ( see ©) 
generates an infinite sequence {t k = n^(£o)}, which can be also defined as 

to=0, =»/(**). fc = 0,l,---. (8) 

As we already observed, this sequence is strictly increasing, remains in [0,t*) 
and converges. 

Corollary 4. The sequence {t k } is well defined, strictly increasing and is con- 
tained in [0,t*). Moreover, it converges Q-linearly to t*, as follows 

U-t k+1 = -^—{U-t k ) 2 <±(U-t k ), fc = 0,l,---. (9) 

If 2bL < 1, then the sequence {t k } converge Q-quadratically as follows 
1-0 2 " L , s0 L 

m ^-t k r, * = o,i 

(10) 



= ; + ^ 2Vi-26L (t '- ffc)2 * i7r=l^ fe)2 ' fc = °' 11 ' ' ' 



where 9 = £*/£** < 1. 
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Proof. The first statement of the corollary have already been proved. 
Using Proposition [3l we have for any k 

which combined with flSJ), yields the equality on ©. As t k € [0, t*), using 
Proposition [5] we have 

f'(tk) < L(t k - U) < 0. 

The multiplication of the first above inequality by (t» — ife)/(2/'(ife)) < yields 
the inequality in ©. 

Now suppose that 2bL < 1 or equivalently t r < t**. A closed expression for 
tk is available ( see, e.g. [[5], Appendix F], [4]) see the Appendix A. In this case 

9 2 " 2V1 - 26i 
tfc = *> - 1 _ g2fc ^ , fc = 0,l, 

From above equation we have that 



t'U \ 1 + 02 ^ /T ^2bI: 

/fa) = - 1 _ ga > fl , * = o,i,.... 

Therefore, to obtain the equality in (|10l) combine the equality in ^ and latter 
equality. As (1 - 6» 2 ")/(l + O 2 ") < 1 the inequality in (JD) follows. □ 

3 Simplifying assumption and convergence 

Newton method is invariant under (non-singular) linear transformations. This 
fact will be used to simplify our analysis. We claim that it is enough to prove 
Theorem Q] for the case X = Y and F'(x ) = I. Indeed, if F'(x ) ^ I, define 

G = F'(x )- 1 F. 

Then, the domain, the roots, the domain of the derivative and the points where 
the derivative is non-singular are the same for F and G. Moreover, Newton 
method applied to F(x) = is equivalent to Newton methods applied to G{x) = 
0, i.e., at the points where F'(x) is nonsingular, 

G'{x)~ 1 G{x) = F'(x)- x F(x), x - G'{x)^ 1 G{x) =x- F'ix^F^x). 

Finally, G will satisfy the same assumptions wich F satisfy. So, from now one 
we assume 

X = Y, F'(x ) = I. (11) 
Note that this assumption simplifies conditions 2 and 3 of Theorem [T] 
Proposition 5. If0<t<t* and x € B(xQ,t), then F'(x) is non-singular and 
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Proof. Recall that t* < 1/L. Hence, < t < 1/L. Using (fTT|) and assumption 
2, with x and xq we have 

H-F'(x)- J|| <Lt<l. 
Hence, using Banach's Lemma, we conclude that F'(x) is non-singular and 

H-F'Wl < V(l- Lt). 

To end the proof, use © to obtain = 1 - Lt for < t < t*. □ 

The error in the first order approximation of F at point x £ int(C) can be 
estimated in any y £ C, whenever the line segment with extreme points x, y 
lays in C . Since balls are convex, we have: 

Proposition 6. If x £ B(xo,R) and y £ B[xo,R] C C, then 
\\F(y) - [F(x) + F'(x)(y - x)] || < ^\\y - x\\ 2 . 

Proof. Define, for 9 £ [0,1], 

y{9) =x + 0(y- x), R{6) = F{y{9)) - [F{x) + F'(x)(y(8) ~ x)] . 

We shall estimate ||i?(l)|j. From Hahn-Banach Theorem, there exists ^ £ X* 
such that 

nen = i, £(jz(i)) = pz(i)ii. 

Define, for 6 £ [0, 1], 

g(e) = c(R(e)). 

Direct calculation yields, for 9 £ [0, 1) 

ljl(8) = z(F'(y(6))-F'(x)). 
In particular, g is C 1 on [0, 1). Using assumption 2, we have 

^L(9)<L9\\y-x\\. 

To end the prove, note that £(i?(0)) = and perform direct integration on the 
above inequality. □ 

Proposition [5] guarantee non-singularity of F' , and so well definedness of 
Newton iteration map for solving F(x) — in B(xo,t*). Let us call Np the 
Newton iteration map (for F(x) ~ 0) in that region 

N F :B(x ,U) -> X 

x ^ x- F'ix^Fix). [ ' 
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One can apply a single Newton iteration on any x £ B(xo,t*) to obtain Np(x) 
which may not belong to 2?(xo,t*), or even may not belong to the domain of 
F. To ensure that Newton iterations may be repeated indefinitely from xq, we 
need some additional results. 

First, define some subsets of B(xo,t*) in which, as we shall prove, Newton 
iteration (TT21 is "well behaved" . 

K(t) := {x £ B[x , t] : ||F(a:)|| < /(*)} , t £ [0, t.) , (13) 

K:= (J (14) 

te[o,t.) 

Lemma 7. For any t £ [0,£*) and x £ K(t), 

1. \\F'(x)^F{x)\\<-f(t)/f'(t), 

2. \\x - N F (x)\\ <n f (t), 

3. \\F{N F {x))\\ < f(n f (t)). 
In particular, 

N F (K(t))cK(n f (t)), Vte[0,t„) 
and Np maps K in K , i.e., Np{K) C K . 

Proof. Take x £ K(t). Using Proposition [5] and (|13l) we conclude that F'(x) is 
non-singular, 

\\F>(x)-i\\<l/\f'(t)\, \\F(x)\\<f(t). 

Hence, 

\\F'(x)- x F(x)\\ < WF'ixy'W \\F(x)\\ </(*)/ 

which combined with the inequality /'(£) < yields item 1. 

To prove item 2 use item 1, triangular inequality and definition (|12l) to obtain 

\\x - JV>(a:)|| < ||i - x\\ + ^(a;)"^^! < t - f(t)/f'(t). 

To end the prove of item 2, combine the above equation with definition (J7J. 
From item 2 and Proposition^ Np(x) £ B(xq, i*). So, Proposition[B]implies 

||F(iV F (a;)) - [F(£)+F'(x) (JV F (a:) - a;)] || < - ll^Or)- 1 ^! 2 . 
Note that by (HU) 

F{x) + F'{x) (N F (x) - x) = 0. 

Combining last two equations, item 1 and identity f(n/(t)) = L(f(t)/f'(t)) 2 /2 
(which follows from ([5]) and (JT))) we conclude that item 3 also holds. 

Since t < n/(t) < t* (Proposition [3]) , using also items 2 and 3 we have that 

N F (x) £ *"(»/(*))■ 

As a; is an arbitrary element of K(t), we have Np(K(t)) C K(nf(t)). 

To prove the last inclusion, take x £ K. Then a; € -K'(i) for some i <E [0,t*), 
which readily implies Np(x) £ K(rif(t)) C K. □ 
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The last inclusion in Lemma [7J shows that for any x 6 K, the sequence 

N° F (x)=x, N f +1 (x)=N f (N f (x)), fe = J l,-.-, 

is well defined and remains in K. The assumptions of Theorem Q] guarantee 

x € #(0) C if. (15) 

Therefore, the sequence {x k — N f (xq)} is well defined and remains in K. This 
sequence can be also defined as 

x Q = 0, Xk+i — N F (x k ), fc = 0, l,---. (16) 

which happens to be the same sequence specified in ©, Theorem [TJ 

Proposition 8. The sequence {xk} is well defined, is contained in B(xo,t*) 
and 

x k €K(t h ), fe = 0,l,-.-. (17) 
Moreover, {xk} converges to a point x* € _B[xo,i*], 

- x k \\ < i* - t k , k = 0, 1,-- ■ , 

and -F(x*) = 0. 

Proof. Well definedness of the sequence {x/,.} was already proved. We also 
conclude that this sequence remains in K. As K C B(xo,t*) (see (fT3|) and 
HI))), {x^} also remains in B(xo,t*). 

As fo = 0, the first inclusion in (JTH]) can also be written as Xq £ K(to). So, 
(jTTJ) holds for k = 0. To complete the proof of (jTT)) use induction in fc, (|l7o)) . 
Proposition [7] and equation ©. 

Combining (IT7|) with item 1 of Lemma [7J (fTB"]) and ([5} we obtain 

\\x k+1 -x k \\<t k+1 -t k , fc = 0,l,---. (18) 

As converges and X]fe°=o — t k < oo we conclude that {x k } is a Cauchy 
sequence. So, {x^} converges to some x* G £?[xo,i*]. Moreover, (fTH]l implies 

oo 

\\x* - Xfc|| < 2j*j+i - tj = U - t k , fc = 0,l,---. (19) 

Note that 

F(x k ) = F'(x fe _i)[x fc _i - x fc ]. 
As ||F'(x)|| is bounded by 1 + Lt* in B(xo,t*) last equation implies that 

lim F(x fe ) = 0. 

k— >oo 

Now, using the continuity of F in B[xo,t*] we have that F(x*) = 0. □ 
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4 Uniqueness and convergence rate 

To prove uniqueness and estimate the convergence rate, another auxiliary result 
will be needed. 

Proposition 9. Take x,y € X, t,v>0. If 

\\x-x \\ <t<U, \\y-x \\ < R, F{y) = 0, f(v)<0 
and B[xq, R] C C , then 

\\y-N F (x)\\<[v~n f (t)}^-^-. 

Proof. Note from ([T2j) 

y - N F (x) = F'ixy^Fix) + F'(x)(y - x)}. 
As F{y) = 0, using also Proposition [5] we obtain 

WF'ixy^Fix) + F'(x)(y - x)}\\ < ±\\y - x\\\ 
and from Proposition [S] 

\\F\xy l \\<i/\f(i)\. 

Combining these equations we have 

As f'(t) < and f(v) < 0, using also © we have 
v-n f (t) = -j—[-f(t)-f(t)(v-t)] 

Combining the two above inequalities we obtain the desired result. □ 
Corollary 10. If y E B[xo,t*] and F(y) = 0, then 



1 < ll_ tk t ^2 \\y~^\\ 2 , \\y-x k \\<u-t k , k = 0,l,-~ 



In particular, the unique zero of F in B[xo,t*]. 

Proof. Take an arbitrary k. From Proposition [8] we have Xk G K{tk). So, 
\\xk ~ x o\\ < tk and we can apply Proposition |H] with x = x k , t — tk and v = 
to obtain 

\\v - N F (x k )\\ < [U - n f (t k )} \l~_ X t % ■ 
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The first inequality now follows from the above inequality, (fTrJ)) and flSJ). 

We will prove the second inequality by induction. For k = this inequality 
holds, because y G B[xo,t*] and t — 0. Now, assume that the inequality holds 
for some k, 

\\y - Xk\\ < t* - t k . 

Combining the above inequality with the first inequality of the corollary, we 
have that \\y — £fc+i|| < t* — £fc+i, wich concludes the induction. 

We already know that x» € B[x ,t r ] and F(x*) — 0. Since {xk} converges 
to x* and {tk} converges to £*, using the second inequality of the corollary we 
conclude y — x*. Therefore, x* is the unique zero of F in B[xq, f*]. □ 

Corollary 11. The sequences {xk} and {tk} satisfy 

\\x*-Xk+i\\<^—!j^\\x*-Xk\\ 2 , fc = 0,l,---. (20) 



In particular, 



|| x* - < ~||x* - Xfc||, fc = 0, (21) 



Additionally, if 2bL < 1 then 

1 — 9 2h L L 
\\x,-x k+l \\ < 1 + ^ 2V1 _ 26L II^-^H 2 ^ 2> /l-2hL l|3! *" a!fc|12 ' * = °' 1j ' ■ 

(22) 

Proof. According to Proposition El x* £ B[xo,t*] and F(x*) = 0. To prove 
equation (|20|) apply Corollary ITUl with y = x*. 

Note that, by ([S]) in Corollary [H and Proposition [51 for any k 

(U - - tk) < 1/2 and ||x* — Xjt||/(i* - ife) < 1. 



Combining these inequalities with (1201) we have (|2lj) . Now, assume that 6L < 
1/2 holds. Then, <JT0j) in Corollary H and (HI imply ((22j) and the corollary is 
proved. □ 

Corollary 12. If2bL < 1, < p < t** and B[xo,p] C C i/ien x* is £/ie unique 
zero of F in B [xo , p] . 

Proof. Assume that there exists y* £ C such that ||y» — x || < p and F(y*) = 0. 
Using Proposition |B] with x = xo and y = y* (recall that -F'(xo) = / ) we obtain 
that 

L 2 
||F(xo)+2/-x || sC ^-||y-x || • 

Triangle inequality and assumption 3 of Theorem [T] yield 

||F(x ) +y-x Q \\>\\y- x || - ||F(x )|| > ||y - x || - 6. 
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Combining the above inequalities we obtain 

■^||y-2:o|| 2 > ||y*-2:o|| - b, 

which is equivalent to f(\\y* — Xq\\) > 0. As |jy* — Xq\\ < p < t„ last inequality 
implies that ||y* — x \\ < i*. Therefore, from Corollary 1101 and assumption 
F(y*) = 0) we conclude that = x*. □ 

Therefore, it follows from Proposition [H Corollary [TUl Corollary [TT] and 
Corollary [T^] that all statements in Theorem [T] are valid. 



4.1 Appendix: A closed formula for tk 

Note that fit) = (L/2)(t - t*)(t - t**), and f'(t) = {L/2)[(t - t*) + (t - i,,)]. 
Using the above equations and ©, 

, , _ f , , s (tk — t*)(tk — <**) {tk—t*) 2 

tfc+i — i* — (Jfc — t*J 



- t*) + - t**) (t fc - t*) + (i fe - t»«) ' 
By similar manipulations, we have 

** (**-**) + (**-*«)' 
Combing two latter equality we obtain that 

/ \ 2 

tk+1 — t* (tk—t* 



tk-\-l ^** \tk t, 

Suppose that 2bL < 1. In this case, t* < Hence, using the definition 

6 := t»/t** < 1, and induction in k we have 



t/c t% 



'J -2 



After some algebraic manipulation in above equality we obtain hat 
t*j 2k - U 2 " 2V1 - 2bL , n , 

tk = TTi = I* ^ , fc = 0, 1, ■ ■ ■ 

2k - 1 1 6» 2fc L 
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